enhancement: add postgres support#1
Merged
Conversation
Signed-off-by: Jason McCallister <jason@mccallister.dev>
There was a problem hiding this comment.
Pull request overview
This PR introduces a pluggable driver architecture for @snaapi/queue and adds a new Postgres-backed implementation (now the default when using the new createQueue() factory), with accompanying docs, tests, and CI updates.
Changes:
- Introduces a
QueueDriverinterface and refactors the queue/worker/dispatcher/middleware pipeline to be backing-store agnostic (Deno KV and Postgres). - Adds a Postgres driver (schema/migrations, LISTEN/NOTIFY + polling consumer, locks/counters/failed-job storage).
- Adds Postgres integration tests, Docker compose setup, and CI job configuration to run both KV and Postgres test suites.
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/queue_test.ts | Updates job context assertions to validate locks/counters instead of kv. |
| tests/postgres_test.ts | Adds Postgres integration test suite covering dispatch, delay, context, middleware, chains, uniqueness, failed jobs, retries, and migrations. |
| src/worker.ts | Refactors worker execution to use QueueDriver and populates JobContext with locks/counters. |
| src/types.ts | Updates JobContext to expose locks/counters primitives; updates comments around backing store. |
| src/queue.ts | Makes Queue accept Deno.Kv or QueueDriver, adds listener lifecycle management, and exposes driver. |
| src/middleware.ts | Migrates rateLimit / withoutOverlapping from direct KV usage to ctx.counters / ctx.locks. |
| src/failed.ts | Refactors FailedJobStore into a driver-backed facade. |
| src/factory.ts | Adds createQueue() factory selecting driver via env/options (defaults to Postgres). |
| src/drivers/types.ts | Introduces driver interfaces: enqueue/listen/unique, failed store, locks, counters. |
| src/drivers/postgres.ts | Adds Postgres driver: schema/migrate, enqueue, LISTEN/NOTIFY listener loop, locks/counters/failed store. |
| src/drivers/deno-kv.ts | Adds Deno KV driver implementing the new driver interfaces. |
| src/dispatcher.ts | Refactors dispatcher to enqueue through QueueDriver (including unique dispatch). |
| src/chain.ts | Refactors chain dispatch/advance to use QueueDriver. |
| scripts/migrate.ts | Adds a script to run Postgres schema migration from the CLI. |
| mod.ts | Exports the new factory and drivers/types. |
| deno.lock | Adds npm:pg dependency. |
| deno.json | Adds imports mapping for pg and adds tasks for Postgres testing and DB lifecycle/migration. |
| compose.yaml | Adds local Postgres service definition for development/testing. |
| README.md | Updates docs to describe backing stores, createQueue(), Postgres usage, and migration notes. |
| .github/workflows/ci.yml | Adds Postgres service and runs both KV and Postgres test tasks in CI. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Reservations now hold the original row until the handler returns, so a mid-flight retry (which preserves envelope.id) collided on the jobs primary key. Generating a fresh UUID for the row keeps the dispatch id on the envelope (and ctx.id) stable while letting retries coexist with the reserved original. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds postgres support, as default, with a new
createQueuefactory helper. PostgreSQL driver run migrations